Method and device for visual range measurements with image sensor systems

ABSTRACT

A method and a device for measuring the visual range using image sensor systems made up of at least two image sensors which record generally the same scene. Objects are detected from the image sensor signals whose distance with respect to the image sensor system is calculated, the object contrast is determined and the visual range is ascertained.

FIELD OF THE INVENTION

The present invention relates to a method and a device for measuring avisual range using image sensor systems made up of at least two imagesensors.

BACKGROUND INFORMATION

In a conventional method, a transmitter emits radiation and thescattered-back portions are detected by a receiver and used to calculatethe visual range. No information regarding the illumination conditionsof the scene and the contrasts of the objects are recorded by thismethod. These parameters play an important role in the visual-rangeperception of humans. As a result, this method is not very suitable forapplications that require information regarding the visual rangeperceived by a person.

As described in International Application No. WO 02/06851, the visualrange may be determined via a method that utilizes an image sensor, bydetecting an object, ascertaining the distance of the object withrespect to the image sensor at two different positions of the object,and determining the individual contrast. The method is limited in thatthe objects must not be moving, and the image sensor itself must be inmotion. Furthermore, the method is unsuitable for image sensors that areoriented in the driving direction, for instance in motor vehicles, sincesmall objects must be detected from a great distance and be tracked upto their approach of the motor vehicle. In this connection, there is theproblem that an object must be selected from a multitude of objectshaving low contrast, without it being known whether this is a suitableobject and one that is large and rich in contrasts once it comes closer.

Another method for determining the visual range is described in EuropeanPatent No. EP 687594 A1. In this method, the presence of fog isdetermined via the ratio of white and black pixels. This method fails inscenes with poor contrast since it erroneously detects fog here.

SUMMARY

The method according to an example embodiment of the present inventionbroadens the functional scope of image sensor systems for ascertainingthe visual range that are made up of at least two image sensors. This isparticularly advantageous in motor vehicle where image sensor systems,in particular those having two image sensors, are utilized indriver-assistance systems to support the driver. There is no longer anyneed to install an additional device in the motor vehicle fordetermining the visual range.

Particularly advantageous is not only the use of the visual rangemeasurement with the aid of image sensor systems in motor vehicles, butalso the use in all image sensor configurations in which at least twoimage sensors record the same scene and already carry out otherfunctions. The application in connection with the monitoring of trafficareas with the aid of image sensor systems is particularly advantageous.By ascertaining the visual range, it is possible here to automaticallyadapt the display of the allowed maximum speed to the visibilityconditions.

In an advantageous manner, the method permits the measuring of thevisual range in the case of moving and non-moving image sensor systems.In motor vehicles, it is possible in this way to determine thevisibility range in all states of motion, in particular also in astationary vehicle.

The visual range may advantageously be determined for static and movingobjects. When using the present invention in motor vehicles, it is thuspossible to determine the visual range in any state of motion of theobjects in the surrounding field of the motor vehicle.

In an especially advantageous manner, the method for determining thevisual range as described here may consider the illumination of thescene. The method may be used in all application cases where a variableof the visual range is required that corresponds to human perception.

According to a first embodiment of the method, the visual range iscalculated in an advantageous manner via arithmetic mean-valuegeneration of at least one individual visual range, which is calculatedfrom the average contrasts of two different distance ranges. This methodis advantageously suitable to calculate the visual range at lowcomputing power of the utilized evaluation unit.

Calculating the visual range by forming an exponential regression of theaverage contrasts over the distance has proven advantageous.

A preprocessing of the image sensor signals may be advantageous. Insituations in which only objects of similar size are located in thevisual range of the image sensors, for instance, the attenuation of thesmall-scale contrasts of far-away objects may lead to an underestimationof the visual range due to the optical characteristics of the imagesensors. This may be prevented by preprocessing the image sensorsignals, in particular by means of high-pass filtering.

In an advantageous manner, the preprocessing of the image sensor signalsmay improve the image quality, for instance by removing imageinterferences.

The visual range calculated from the image sensor signals mayadvantageously be utilized in downstream systems. For example, indriver-assistance systems in motor vehicles, an optical, acousticaland/or haptic warning of the driver is possible once a maximum speedderived from the visual conditions has been exceeded.

In particular the turning on of fog lights and/or the low beam ispossible in motor vehicles in the event that a minimum visual range isnot attained.

Further advantages will become apparent from the following descriptionof exemplary embodiments with reference to the figures.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, the present invention is explained in greater detailin light of the specific example embodiment shown in the figures.

FIG. 1 shows block diagram of the device for measuring the visual range.

FIG. 2 shows a flow chart of the method for determining the visual range28 from image sensor signals 21 and 22.

FIG. 3 shows the average contrast {overscore (C)} (x) of objects as afunction of distance x, a regression curve 31 and the width of adistance range Δx.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

FIG. 1 shows a block diagram of the device made up of an image sensorsystem having an image sensor 11 and a second image sensor 12, twoimage-sensor signal lines 13 and 14, an evaluation unit 15, anoutput-signal line 16 and a downstream system 17.

CCD or CMOS cameras, for instance, are able to be utilized as imagesensors. Both image sensors are arranged such that they image the samescene, but from a slightly different viewing angle. The image sensorstransmit images of the monitored scene to evaluation unit 15. Evaluationunit 15 generates a signal of the measured value of the visual range onoutput-signal line 16. This output signal is transmitted electronically,digitally, acoustically and/or visually to at least one downstreamsystem 17 for display, information and/or storing. Evaluation unit 15 ismade up of a plurality of modules 23, 24, 25, 26 and 27 shown in FIG. 2,which are configured as programs of at least one microprocessor in thepreferred exemplary embodiment.

The method is based on a statistical characteristic of natural scenesaccording to which the probability of an object type occurring isindependent of its distance from the image sensor system. In the method,the characteristic of objects i is used that their object contrast c₁,defined as the contrast measured at distance 0, is statisticallyindependent of the position of the object relative to the image sensorsystem. On statistical average, one can then observe in each distancerange x of the image sensor system across all n objects located in thisdistance range the same average object contrast {overscore (c)}:$\begin{matrix}{\overset{\_}{c} = {{\frac{1}{n}{\sum\limits_{i - 1}^{n}{c_{1}(x)}}} = {{const}.}}} & (1)\end{matrix}$

The average object contrast c thus is independent of the distance fromthe image sensor system on statistical average.

The observable contrast C_(i) (x) of an individual object i havingobject contrast cl drops with increasing distance, according toLambert's law, as a function of visual range D: $\begin{matrix}{{C_{i}(x)} = {c_{i}{\mathbb{e}}^{- \frac{x}{D}}}} & (2)\end{matrix}$

For average contrast {overscore (C)} (x) in a certain distance range xacross a plurality of objects located in this distance range and whichwere ascertained within a time window, it holds: $\begin{matrix}{{\overset{\_}{C}(x)} = {\frac{1}{n}{\sum\limits_{i - 1}^{n}{C_{i}(x)}}}} & (3)\end{matrix}$

On the basis of (1) and (2), for the calculation of average contrastC(x) it holds according to (3): $\begin{matrix}{{\overset{\_}{C}(x)} = {\overset{\_}{c}\quad{\mathbb{e}}^{- \frac{x}{D}}}} & (4)\end{matrix}$

Average contrast {overscore (C)} (x) thereby follows the sameexponential law as the observable contrast C₁ (x) of an individualobject i.

If measurements of the average contrast for two distance ranges x₁ andx₂ are available, the visual range results as follows: $\begin{matrix}{D = \frac{x_{2} - x_{1}}{{\ln\quad{\overset{\_}{C}\left( x_{1} \right)}} - {\ln\quad{\overset{\_}{C}\left( x_{3} \right)}}}} & (5)\end{matrix}$

The longer the duration during which the average contrast is measuredfor a particular distance range and the more objects are present in thescene, the smaller the measuring error in the visual rangedetermination.

FIG. 2 shows a flow chart for implementing the method in a schematicrepresentation. Image sensor signals 21 and 22 of signal lines 13 and 14are supplied to preprocessing modules 23 and 24.

A distance and contrast measurement is implemented in module 25. There,in the first step, objects that are completely within the visual rangeof both image sensors are detected using image processing methods viaimage segmentation. In the second step, the distance of the object fromthe image sensor system is ascertained. An especially suitablepossibility for distance measuring are block-based stereo methods. Here,the distance of the objects is measured via the correlation of imageblocks along the epipolar in both images. The distance of the objectfrom the image sensor system may be calculated from the relativedisplacement of the image blocks in both images, since the distance isinversely proportional to the displacement of the image blocks.

In step three, the contrast of the objects will then be determined. Inthe preferred exemplary embodiment, the contrast is calculated by way ofthe amount integral of a cut-off filter across the image block. Othermethods known from image processing are possible for calculating thecontrast over an image detail, for example the calculation of thestandard deviation and the variance of the gray-scale values within animage block.

The method assigns a distance value to each ascertained contrast value.The value pairs thus obtained are transmitted as object characteristicdata to downstream module 26 for further processing.

In module 26, the calculation of the average contrast in each distancerange is calculated. To this end, the distance is divided into distanceranges. The distance ranges in the preferred exemplary embodiment arecharacterized by the same width Δx. In a modified method, it is possibleto adapt the width of the distance ranges as a function of variousparameters, such as time, distance and/or movement state of the imagesensor system. The object characteristic data ascertained in module 25are assigned to the distance ranges. The classification is implementedon the basis of the distance parameter. Within each distance range theaverage contrast is calculated by mean value generation according toformula (3). The generation of the average contrast is based on objectcharacteristic data, which were ascertained inside a time window priorto the calculation instant. The time window is to be selected such thatthe visual range does not change significantly inside the window. Theaverage contrasts of each distance range are transmitted to downstreammodule 27 for calculation of the visual range. Two calculation variantsare possible for calculating the visual range. According to a firstvariant, the visual range is formed by arithmetical mean valuegeneration of at least one individual visual range. The individualvisual ranges, such as D(x₃, X₄), are formed from average contrastvalues of in each case two different distances, here, X_(3 and X) ₄,according to formula (5).

According to a second variant, the visual range may be calculated via anexponential regression, drawn in as regression curve 31 in FIG. 3.

Output signal 28, which is a measure of the visual range, is transmittedvia signal line 16 in FIG. 1 to downstream system 17.

FIG. 3 shows the average contrast {overscore (C )} (x) of objects in thesame distance range as a function of distance x. The width of a distancerange Δx has been drawn in.

In situations in which only objects of similar size are located withinthe visual range of the image sensors, the attenuation of thesmall-scale contrasts of far-away objects could lead to anunderestimation of the visual range due to the optical properties of theimage sensors. This is able to be prevented by a preprocessing of theimage sensor signals in preprocessing modules 23 and 24, in particularby high-pass filtering. As an alternative, it is possible to reduce thiserror by a corresponding adaptation of the exponential regression curve.In addition, preprocessing modules 23 and 24 may be used to improve theimage quality, for instance to remove interference, to improve contrastand/or to sharpen the edge.

The ascertained visual range is transmitted in a suitable manner to atleast one downstream system 17. For instance, an adaptation of at leastone system on the basis of the visual range is conceivable and/or thedeenergizing or energizing of at least one system when leaving anadjustable value range of the visual range. An application possibilityresults in driver-assistance systems in motor vehicles. Here, anoptical, acoustical and/or haptic warning of the driver is conceivablewhen a maximum speed derived from the visual conditions is exceeded. Themethod is particularly suited for turning on the fog lights and/or thelow beam in motor vehicles when a minimum visual range is not attained.The method may preferably be used to deactivate a distance warningsystem in motor vehicles based on an image sensor system when a minimumrange of vision is not attained.

The method and the device described are not limited to the use ofimage-processing sensor systems made up of two image-processing sensorsin a motor vehicle. With systems having more than two cameras, thevisual range may in each case be generated from two image sensorsignals. By using statistical methods, the measuring error for thecalculated visual range may be reduced. A prerequisite is merely thatthe image-processing sensors utilized record the same scene.

Furthermore, the described procedure with the corresponding features maybe utilized outside of motor vehicle technology. The use in image sensorsystems for monitoring traffic spaces comes to mind as applicationexample. For instance, the method may be used for the automaticadaptation of the display of the permitted top speed to the visualconditions and/or for generating a fog warning system for trafficparticipants.

1-17. (canceled)
 18. A method for measuring a visual range using animage sensor system including at least two image sensors, the methodcomprising: recording, via the at least two image sensors, a same scene;ascertaining from image sensors signals from the at least two sensors afirst variable which represents a contrast of an image or image detailof the recorded scene; ascertaining a second variable as a function ofthe image sensor signals from the at least two image sensors, the secondvariable representing a distance with respect to a recorded object inthe image or the image detail of the recorded scene; and determining thevisual range as a function of the first and second variables.
 19. Themethod as recited in claim 18, wherein the image sensor system includestwo image sensors.
 20. The method as recited in claim 18, furthercomprising: detecting objects in an image area of the at least two imagesensors; determining a distance of each of the detected objects withrespect to the image sensor system; ascertaining a contrast of each ofthe detected objects; processing object characteristic data, the objectcharacteristic data including the determined distance and theascertained contrasts; and calculating the visual range based on theobject characteristic data.
 21. The method as recited in claim 20,wherein the processing of the object characteristic data includes:assigning the distance and the contrast of each of the objects to avalue pair; classifying the value pairs of the objects based on thedistance, and subdividing into distance ranges; and calculating anaverage contrast in each of the distance ranges.
 22. The method asrecited in claim 21, wherein the width of each of the distance ranges isthe same.
 23. The method as recited in claim 21, wherein the width of atleast one of the distance ranges is adapted as a function of at leastone of time, distance, and state of motion of the image sensor system.24. The method as recited in claim 20, wherein the distance iscalculated from a relative displacement of two corresponding blocks ofan object in spatially corresponding images of the two image sensors.25. The method as recited in claim 18, wherein the contrast of theobject in the image or the image detail is formed by an amount integralof a cut-off filter over a selected detail of the image of at least oneimage of the two image sensors.
 26. The method as recited in claim 21,wherein at least one individual visual range is calculated from averagecontrast values of in each case two different distance ranges withrespect to at least one distance range.
 27. The method as recited inclaim 26, wherein the visual range is formed by subsequent arithmeticalmean value generation from at least one individual visual range.
 28. Themethod as recited in claim 21, wherein the visual range is calculated byan exponential regression of the average contrasts over the distance.29. The method as recited in claim 18, wherein the image sensor signalsare preprocessed.
 30. The method as recited in claim 18, wherein thedetermined visual range is used for at least one of: i) adapting atleast one downstream system, and ii) energizing or de-energizing atleast one system.
 31. The method as recited in claim 30, wherein the atleast one system is energized or de-energized when an adjustable valuerange of the visual range is left.
 32. The method as recited in claim18, wherein the image sensor system is located in a motor vehicle.
 33. Adevice for measuring a visual range, comprising: an image sensor systemincluding at least two image sensors that record a same scene; and anevaluation unit configured to calculate the visual range from the atleast two image sensor signals and to generate an output signal as ameasured value which is a measure for the visual range.
 34. The deviceas recited in claim 33, wherein the evaluation unit is configured toperform the following steps: detecting objects in an image area of theat least two image sensors; determining a distance of each of thedetected objects with respect to the image sensor system; ascertaining acontrast of each of the detected objects; processing objectcharacteristic data, the object characteristic data including thedetermined distance and the ascertained contrasts; and calculating thevisual range based on the object characteristic data.
 35. The device asrecited in claim 15, wherein the image sensor system is at least one of:made up of two image sensors, and located inside the motor vehicle.